fix(review): require reachability evidence for blockers - #93
Conversation
Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Codex <noreply@openai.com>
🤖 Codex PR Review🚫 Merge blocked: 1 serious issue(s) found in high-risk files 🚫 Blocking IssuesThese issues must be fixed before this PR can be merged: 1. 🟠 [HIGH] Logic in
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b046033d32
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Codex <noreply@openai.com>
Goal Driver reachability arbitrationCurrent head:
Required |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd432fead4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 5. Do not invent backward-compatibility requirements that are absent from the repository and PR contract. When the repository and PR explicitly define a clean-slate namespace with legacy compatibility out of scope, review that boundary for accidental fallback instead of requesting dual-read or migration. This never overrides security or data-integrity findings. | ||
| 6. For public JSON/wire contracts, systematically check optional-key presence versus explicit null, recursive JSON-safe types, every identity-bearing integer range, one canonical timestamp representation, deterministic encode/decode round-trips and digests, deep immutability, and identifier/path safety. | ||
| 7. For each finding, classify its severity: | ||
| 4. Assign **critical** or **high** only when the supplied PR context proves all of the following: an exact changed path and line; a current caller or entry point proven by the supplied PR context, whether pre-existing or introduced by this PR, or an explicitly declared public untrusted boundary; reachability under the current configuration and inputs; and a concrete correctness, security, or data-integrity impact. State that reachability and impact in the finding description. If any element is missing, downgrade it to medium or low or omit it. |
There was a problem hiding this comment.
Keep crash bugs eligible for blocker severity
For PRs where the proven impact is a reachable production crash, this new gate allows critical/high only for correctness, security, or data-integrity impact and then says to downgrade or omit when any element is missing. That conflicts with the prompt’s own crash-risk priority and critical severity definition, and because evaluate_findings blocks solely on critical/high, a concrete null dereference or resource-exhaustion regression can now become non-blocking. Include crash/reliability impact in the allowed blocker criteria.
Useful? React with 👍 / 👎.
| 5. Do not invent backward-compatibility requirements that are absent from the repository and PR contract. When the repository and PR explicitly define a clean-slate namespace with legacy compatibility out of scope, review that boundary for accidental fallback instead of requesting dual-read or migration. This never overrides security or data-integrity findings. | ||
| 6. For public JSON/wire contracts, systematically check optional-key presence versus explicit null, recursive JSON-safe types, every identity-bearing integer range, one canonical timestamp representation, deterministic encode/decode round-trips and digests, deep immutability, and identifier/path safety. | ||
| 7. For each finding, classify its severity: | ||
| 4. Assign **critical** or **high** only when the supplied PR context proves all of the following: an exact changed path and line; a current caller or entry point proven by the supplied PR context, whether pre-existing or introduced by this PR, or an explicitly declared public untrusted boundary; reachability under the current configuration and inputs; and a concrete correctness, security, or data-integrity impact. State that reachability and impact in the finding description. If any element is missing, downgrade it to medium or low or omit it. |
There was a problem hiding this comment.
Keep API breaks eligible for blocker severity
When the supplied diff itself proves a changed public entry point or config format, this rule still prevents high unless the reviewer can frame the impact as correctness, security, or data-integrity. A signature/schema/config-format break is explicitly listed as a review priority and high-severity example below, but under this new gate the model can downgrade it, and evaluate_findings then lets the PR pass despite a reachable downstream compatibility break.
Useful? React with 👍 / 👎.
| 5. Do not invent backward-compatibility requirements that are absent from the repository and PR contract. When the repository and PR explicitly define a clean-slate namespace with legacy compatibility out of scope, review that boundary for accidental fallback instead of requesting dual-read or migration. This never overrides security or data-integrity findings. | ||
| 6. For public JSON/wire contracts, systematically check optional-key presence versus explicit null, recursive JSON-safe types, every identity-bearing integer range, one canonical timestamp representation, deterministic encode/decode round-trips and digests, deep immutability, and identifier/path safety. | ||
| 7. For each finding, classify its severity: | ||
| 4. Assign **critical** or **high** only when the supplied PR context proves all of the following: an exact changed path and line; a current caller or entry point proven by the supplied PR context, whether pre-existing or introduced by this PR, or an explicitly declared public untrusted boundary; reachability under the current configuration and inputs; and a concrete correctness, security, or data-integrity impact. State that reachability and impact in the finding description. If any element is missing, downgrade it to medium or low or omit it. |
There was a problem hiding this comment.
Do not require callers for credential leaks
For PRs that add a plaintext token or secret to a tracked file, the security impact is the exposure in the diff itself, not a reachable runtime caller or public input boundary. This new rule still requires a current caller/entry point or declared public untrusted boundary before allowing critical/high, so a credential leak can be downgraded to non-blocking even though credential leaks are the top review priority and evaluate_findings only blocks those two severities.
Useful? React with 👍 / 👎.
Summary
Scope
Prompt, prompt-construction text, focused tests, and architecture documentation only. No parser, evidence store, event persistence, workflow, permission, deployment, or runtime changes.
Verification
python3 -m pytest -q tests/test_run_codex_pr_review.py-> 57 passedpython3 -m pytest -q-> 655 passed, 1 skipped, 50 subtestspython3 -m compileall -q scripts testsactionlintgit diff --checkruffunavailable locallyReview contract
One tests-first concentrated closure maximum. Findings that do not prove a current caller/public boundary and reachability are advisory, not blockers.